A Virtual Meeting Service (VMS) is the concept of creating a multimedia meeting room environment virtually within an electronic network by bringing a group of people together without leaving their offices. A VMS allows participants to conference from their desktops using any of the audio, image, video and data combinations available at their conferencing station so that each has the capabilities he or she would otherwise have in a real face-to-face meeting.
Since VMS is a multi-point, multimedia communications service, bridging of audio, video and data streams is of paramount importance to the service. U.S. Pat. No. 5,471,318 to Ahuja et al. dated Nov. 28, 1995, (the '318 patent) describes a circuit configuration in a multimedia network representing a controllably persistent virtual meeting room which facilitates providing VMS.
As indicated in the '318 patent, prior approaches to these conferencing systems involved a plurality of complicated and expensive computer conferencing stations connected together by transmission facilities which act merely as a transport mechanism for whatever is to be communicated between conferencing stations. Each of the conferencing stations in these prior distributed environments contained a large amount of the multimedia data processing capabilities, including the necessary bridging and synchronization circuits.
The '318 patent provided an alternative to distributed environments, enabling a telecommunications carrier to provide a multimedia communications service through a shared multimedia work environment accessed by participants from a plurality of multimedia ports. By implementing the invention of the '318 patent, the bulk of the multimedia data processing capabilities, including the necessary bridging and synchronization circuits, become part of the communications network transmission facilities.
With establishment of a virtual meeting room using VMS, the conferencing stations of the participants in a virtual meeting room require the visual images to be displayed be satisfactorily organized on each conferencing station video screen. "Images" embraces any number of concepts but those routinely expected are the viewing of the participants in the virtual meeting room, the running of software applications programs which have image display, and a "chalk board" on which participants provide notes, annotations and the like.
A problem with conventional video-conferencing imaging is that participant image display is limited to displaying either a single, and the same, participant on every participant's video screen, or is limited to displaying only up to four participants, again all the same, on every participant's video screen. In conventional video-conferencing systems, typically, a codec and a transmission line is required for each participant to transmit into the video conference and a combiner and positioner is required for each participant to receive the various transmissions and combine the text/graphics computer output with the video which are then displayed visually on the receiving participant's video screen.
A problem with conventional systems is that when more than one participant needs to be displayed, multiple codecs and multiple transmission lines are required.
The inventors have observed that if there were a way to successfully combine the video streams from the different participants into a single video stream, this would markedly reduce the number of lines and hardware required since the bandwidth limitation of the transmission line remains constant regardless whether a single video stream or multiple video streams are being transmitted. The problem is how to maintain the image quality for the various video streams in a combined state with a minimum of equipment.
It is known that video streams can be combined in a "quad" box. A quad box takes four video streams and combines them into one video stream which can then be displayed on the video display screen. However, the quad box requires that an aspect ratio of height to width for each image of 3:4 be maintained causing the four-quadrant video image to take up the majority of the screen real estate and is typically centered in the screen as shown in FIG. 1.
A quad box approach may be an acceptable tool for "video and audio only" teleconferences where four participants only want to see one another. However, VMS is intended to be a multimedia teleconference approach, with the purpose being to permit participants to collaborate by passing information back and forth and have that information simultaneously displayed on each participant's screen. Unfortunately, a problem with the quad box is that it does not permit sufficient screen real estate to be available for data display, for example, to exchange text and provide markups or initiate a financial spreadsheet to which each participant can then contribute. Consequently, a quad box unsatisfactorily restricts the unbroken screen real estate which is necessary for use as a graphical or textual communications surface as part of VMS.
There is therefore a need in the art for a video conferencing screen display which provides composite video images of the participants, which is less limited as to the number of participants, which is customizable by each participant, and which is economical in usage of screen real estate thereby providing sufficient unencumbered screen real estate for text and graphics collaboration among participants. Moreover, there is a need for a solution which can augment VMS and which can be centrally based rather than distributed among participants.
Solution
These problems are solved and a major advance over the prior art is achieved by the instant invention which implements an analog approach to composite image display of participants.
A novel approach to ganging quad boxes within a communications network has been discovered by the inventors. Under this approach, a network server includes an Analog Video Bridge connected to an Analog Custom Composite Module. The Analog Custom Composite Module incorporates and uses multiple quad boxes to receive and moderate the video input streams of the participants. The multiple quad boxes are ganged and controlled by the Analog Custom Composite Module in such a fashion that the video stream of each participant can be selectively input to a specific input port on a specific quad box and the output video stream from each quad box can be selectively provided as input to another quad box, in effect controllably cascading and combining each video image with others to reduce the size of each video image and control the location of each video image on each participant's screen.
Consider, as a example, that a first quad box combines up to four video input streams into a single video input stream and the combined image stream so generated would normally be displayed in four quadrants on a receiving video screen occupying the balance of the screen real estate as shown in FIG. 1. However, applying the instant invention, the combined image stream generated as output from the first quad box can be combined with another image stream generated as output by a second quad box by sending both image streams to a third quad box. The third quad box combines the image streams from the first and second quad boxes into a single combined image stream and this twice-combined stream is then sent to the selecting participant. This ganging arrangement permits each quadrant of a four quadrant display to be split into four quadrants as well. Thus, up to sixteen participants can be displayed at a time on a participant's screen.
Normally, this display would occupy the bulk of video screen real estate. However, the present invention novelly recognizes the combination of the combined video stream with Chroma Keying which permits display boxes which are not being used to be turned off and this screen real estate used to display other data.
Further increases in the number of participant's images displayed and further reduction in the size of images are possible by incorporating additional quad boxes. For example, further serial ganging to a fourth quad box will produce images which are 1/64th screen size. However, because image quality is affected by image size and the number of screen pixels used to display the image, continued quad box ganging may not produce a sufficiently clear image for easy viewing.
Moreover, the instant invention permits the image size of a participant to be different form the image size of other participants. This is accomplished by changing the number of quad boxes that one or more of the participants' images are cascaded through relative to the number of quad boxes one or more other participants' images are cascaded through. For example, a first participant's image can be shown in 1/4th size while other participants are shown in 1/16th size by introducing the image streams for the other participants as input to a first quad box and introducing the combined video stream generated by the first quad box as input to a second quad box which is the first quad box to which the image stream of the first participant is introduced as input. This will result in the other participants being displayed on the screen in 1/16th size and the first participant being shown in 1/4th size. This configuration limits to twelve the number of other participants. Similarly, other differences in size can be achieved by adding one or more additional cascades for selected participants through additional quad boxes permitting some participants to be shown as large as 1/4th size while others could, for example, be shown in 1/16th size and still others shown in 1/64th size.
The input port to which a participant image is directed on each quad box used in the custom composition device determines the relative placement of that participant's image with respect to all other participants' images and with respect to the left and right and up and down orientation of the display screen real estate. Because an input port of a quad box will always display the image input to that port in a specific and determined quadrant of the four quadrants produced by the quad box and because those four quadrant displays sent as an input stream to a second quad box will always occupy a specific and determined quadrant of the four quadrants produced by the second quad box and so forth, an image can be selectively directed to appear in a desired image box on the display screen. Thus the instant invention is not only able to control image size but is also able to control image location on the display screen.
In accordance with one aspect of our invention, in a VMS environment, an custom composition device comprised of multiple quad boxes is provided centrally in the communication network to selectively control the size and location of each participant image on a conference station screen.
In accordance with another aspect of our invention, initial participant image streams are sent within the custom composition device to a first quad box location device and the combined image streams created at the first quad box location are sent to a second quad box location whereby each participant's image in the combined image stream output is reduced to approximately 1/16th original size.
In accordance with a further aspect of our invention, the combined image streams created at the second quad box location are sent to a third quad box location whereby each participant's image in the combined image stream output is reduced to approximately 1/64th original size.
In accordance with a still further aspect of our invention, the reduction in a participant's image size is determined and controlled by the number of quad boxes through which the participant's image stream is serially passed.
In accordance with yet another aspect of our invention, the location of a participant's image displayed on a screen is controllably determined by the input port at which the image stream, in combined and uncombined form, is serially presented to the quad boxes 1-n before being displayed.
In accordance with a yet further aspect of our invention, the size of one or more participants' images can be controllably varied from the size of one or more other participants' images by selectively varying the number of quad boxes to which one or more participants image streams are presented relative to the number of quad boxes to which one or more other participants image streams are presented.